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Abstract 

In this paper, we consider a zero error coordination problem wherein the nodes of a network 
exchange messages to be able to perfectly coordinate their actions with the individual observations 
of each other. While previous works on coordination commonly assume an asymptotically vanishing 
error, we assume exact, zero error coordination. Furthermore, unlike previous works that employ 
the empirical or strong notions of coordination, we define and use a notion of set coordination. This 
notion of coordination bears similarities with the empirical notion of coordination. We observe that 
set coordination, in its special case of two nodes with a one-way communication link is equivalent 
with the “Hide and Seek” source coding problem of McEliece and Posner. The Hide and Seek 
problem has known intimate connections with graph entropy, rate distortion theory, Renyi mutual 
information and even error exponents. Other special cases of the set coordination problem relate 
to Witsenhausen’s zero error rate and the distributed computation problem. These connections 
motivate a better understanding of set coordination, its connections with empirical coordination, 
and its study in more general setups. This paper takes a first step in this direction by proving new 
results for two node networks. 


1 Introduction 

Consider a network where each node of the network has a private observation and needs to pro¬ 
duce an action. These actions should be coordinated with the observations; therefore some form 
of communication is necessary among the nodes. The fundamental limits of the required communi¬ 
cation was originally studied by Cuff et al. in [ 1 ] where the authors assumed that the observation 
of node i {1 < i < m) are i.i.d. repetitions of some random variable Aj. The joint distribution 
of (Xi, X2,Xm) was assumed to be a given. Denoting the action of the i-th node by U, co¬ 
ordination was then modeled as requiring the joint pmf of the outputs conditioned on the inputs 
to be very close to some given p{yi,y2, .■.,ym\xi,X2, Here the authors introduce two no¬ 

tions of empirical and strong coordination; in the strong coordination, memoryless repetitions of the 
channel p{yi,y2, ■■■,ym\xi,X2, ■■■,Xm) are simulated, whereas in empirical coordination, only the data 
histograms (or its joint type) is equal p{yi,y2, ■■■,ym,xi, X2, ■■■,Xm)- The common theme is that the 
conditional pmf p(?/i, 7/2, ym\xi,X2, ■■■, Xm) is approximated asymptotically as the number of i.i.d. ob¬ 
servations (the block length) goes to infinity. In this work, however, we are interested in exact zero 
error coordination, i.e., coordination should be achieved with probability one. In this way, our work 
is related to [a Sec IV], [ 3 ] on exact strong coordination capacity, however we adopt a different set 
coordination criterion (which is closer to the empirical notion of coordination). 

In our setup, for any observation vector {xi,X2, ...,Xm) by the m nodes of a network, we assume 
a permissible set of output actions {yi,y2, ■■■,ym)- In other words, we are not directly interested in 
simulating a given 

p{yi,y2, ■■■,ym\xi,x2, ...,Xm)- 
Rather, for every {xi,X2, ...,Xm), we define a set 

dl.xi,X 2 ,...,Xm — Tl ^ T2 X ■ ■ ■ X Tm; 
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such that {yi,y 2 , G •Axi,x 2 ,...,xm- Here 3^* is the action set of node i. We call this a “set 

coordination.” In Section [3.11 we compare set coordination with empirical coordination. 

Example 1. If \Axi,x 2 ,...,xm\ — 1? value of {yi,y 2 , ■■■,ym) will he uniquely specified and will he a 
deterministic function of {xi,X 2 , In this case, coordination reduces to distributed computation. 

Distributed computation is itself a more general problem that the message transmission problem, since 
the functions computed by the nodes can be taken to be the message of other nodes. 

Consider the special case of a network with two nodes with node one has input Xi and node two 
producing output Y 2 . We assume that the input of node two, X 2 , and the output of node one, Yi, 
are disabled, i.e., \X 2 \ = |3^i| = 1. Then for every xi we have a set Ax^ C T 2 - Assume a one-way 
communication link from node one to node two. The goal of the first user will be to send a message 
from node one to node two that will enable production of 7/2 € Ax^ at node two. We show in Section [5] 
that this special case of the coordination problem is equivalent with the “Hide and Seek” problem of 
McEliece and Posner [3]. McEliece and Posner define a source coding problem and a zero-sum “Hide 
and Seek” game. Rather surprisingly, they illustrate that the optimal compression rate of the source 
coding problem can be expressed in terms of the Nash equilibrium of the game. Additional insight 
was provided by Lovasz who provided an elegant combinatorial argument for the result of McEliece 
and Posner in [6]. We review other related results and in particular connections with Renyi mutual 
information of order a is discussed in Section 15.2.21 

We continue by two examples that illustrate connections with zero error rate distortion (see [3 
Ch. 2], [8]), and with graph entropy. Eirstly, consider a non-negative distortion function satisfying 
d{xi,y 2 ) = 0 if and only if 7/2 € Axi- Then, coordination is equivalent with zero distortion in 
reconstruction. Secondly, consider the source coding problem for a source that is taking values in a 
set Ai. We are given a graph ^ on Ai, where two symbols xi and x'^ are connected to each other if 
it is legitimate to reconstruct x'^ when the source value is xi. We can model this by assuming that 
3^2 = Ai and Ax^ being equal to the set of all x'^ that are connected to xi. The message transmitted 
from node one to node two represents the compressed message. The minimum compression rate in 
this case is equal to the logarithm of the chromatic number of the complement of Q in the one-shot 
case when only one instance of the source is given. After coloring the vertices of the complement of Q, 
the message can be the identity of the color that is assigned to the source symbol. In the asymptotic 
case when multiple instances of the source are observed, the answer is the logarithm of the fractional 
chromatic number [9l p. 2215]. 

For the asymptotically vanishing error model, authors in [H Conjecture 1] conjectured that empiri¬ 
cal coordination and strong coordination have the same rate regions when infinite common randomness 
is provided to the parties. Considering a special two node network, we observe connections in the zero 
error model. In [3], authors considers strong coordination with unlimited common randomness and ar¬ 
rives at expressions that match the one given by McEliece and Posner. However, the work of McEliece 
and Posner (or its follow up works) are not cited in [3] and the connection is not noted. See Section 
15.31 for more details. 

This paper is organized as follows: in Section [2] we set up the notation that we use. Section 
o defines set coordination and defines one-way coordination capacity for a two node problem. A 
general lower bound for this problem is given in Section 01 Section [5] provides a detailed treatment 
for the special case of two nodes and discusses its connections with various known results. Section [6] 
computes the coordination capacity when the side information of the second node is a function of the 
side information of the first node. Finally, in Section (3 we consider linear coordination and provide 
several new results. Extensions to MAC and BC setups are given in Section 17.41 

2 Notation and Preliminaries 

We adopt the notation of m- In particular, we show the set {1,2, ...,m} by [ 777 ], and the set {k -|- 
1, A: -|- 2,..., m} by [A: -|- 1 : m]. All random variables in this paper are finite discrete random variables. 
All the logarithms are in base 2 in this paper. 
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Given two graphs Qi,Q 2 , the tensor product Q\ (g) C /2 is a graph whose vertex set is the Cartesian 
product of the vertex sets of Q\ and Q 2 defined as follows: two vertices (mi,U 2 ) and (t'i,U 2 ) in are 
adjacent in 0 G 2 if and only if ui is adjacent with vi in Gi and U 2 is adjacent with V 2 in Gi ■ 

There are many definitions for Renyi mutual information (see m for a review). One dehnition 
for R&yi mutual information of a joint pmf p{x, y) is as follows: 

Ia{X-,Y) =mmDa{p{x,y)\\p{x)q{y)), (1) 

i{y) 

Where Da is the Renyi divergence between two pmfs is defined as follows: 


Da{p\\q) 



log 



Note that as a converges to one, Renyi divergence and Renyi mutual information of order a tend to 
the KL divergence and Shannon’s mutual information. 

In [121 Eq 13], it is shown that mutual information of order a, as defined in equation ([1]), is equal 


to: 


Ia{X;Y) 


a 

a — 1 



y X 



(2) 


R&yi mutual information Ia{X;Y) is a non-decreasing function of a for a E [0, 00 ] [131 Thm 3]. 

We now provide explicit expressions for Ia{X] Y) when a = 0 and a = 00 . When a goes to zero, 
YlxPi^)piy\^)^ to power 1/a which goes to infinity; thus only the largest term is important. In 
fact, one can show that 


Io{X-,Y) = Im 


a 


a^o a — 1 
1 

im — 

a 


log ( max 
y 


['^p{x)p{y\^^ 


1/a 


lim-log ( max \ p(x)p{y\xY ]) 

a — 1 V y V 


= —log(max y P{x))- 

71 / ^ 


p{y\x)>0 


( 3 ) 


Similarly, one can show that as a tends to inhnity, we have 


Iaa{X-,Y)=log 



max p{y\x) 

x-.p(x)>0 


( 4 ) 


3 System Model 

3.1 Set Coordination 

Definition 1. Given an input pmf p{xi, ■ ■ ■ ,Xm) and action sets Axi,x 2 ,..;X„i, one-shot and asymp¬ 
totic coordination are defined as follows: in one-shot coordination, the parties observe only one in¬ 
stance of Xi and coordination is achieved if {yi,y 2 , ■■■,ym) G Axi,x 2 ,...,xm f^i" any {xi,--- ,Xm) where 
p{xi,--- ,Xm) > 0. In the asymptotic version, the parties observe n i.i.d. repetitions of the sources 
Xf,Xlf, ■ ■ ■ ,Xff. Coordination is achieved if iyii,y 2 i, ■■■,ymi) € Axii,x 2 i,...,x„,i for any i E [n] and any 
{xf, ■■■ , x” ) where p{xf, ■■■ , x” ) = 0"=! > ^rni) > 0. 

We assume that the nodes have access to limited communication resources, as well as possibly 
private or common randomness. However, similar to empirical coordination, without loss of generality 
we can assume that the nodes are deterministic and do not use shared or private randomness. This 
is because {yi,y 2 , ■■■,ym) G Axi,x 2 ,...,xm with probability one, and hence it has to hold for all possible 
values of the shared randomness variable. 
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Figure 1: Coordination using a one-way communication link. 


Set coordination is related to empirical coordination. Take some arbitrary conditional pmf 


p{yi,y2, ...,ym\xi,X 2 


such that p{yi,...,ym\xi,...,Xm) > 0 only if (?/i, ?/ 2 , 2/m) € Then a zero-error empirical 

coordination code for p{yi,y 2 , ■■■,ym\xi, X 2 , ■■■, Xm) is also a zero-error set coordination code. How¬ 
ever, set coordination is more relaxed in the asymptotic formulation. Take a set coordination code 
of block length n and two sequence - ,x^) of the same type. Let 

( 2 / 1 ) 2/2 ) ■ ■ ■ ) 2/m) ( 2 / 1 ) 2/2 ) ■ ■ ■ > 2/m) denote the actions of the nodes in response to (x”, X 2 , • • • , x'^) 

and (x”,X 2 ,--- ,x^) respectively. Then it can be the case that the joint types of the sequences 
{xi, X 2 , • • • , a^m; 2 / 1 , 2/2 ! • • • , 2/m) and (xj", x^, • • • , x” , yj), • • • , y” ) are different. Therefore, one can¬ 
not assign a single empirical conditional type p(yi, ...,ym\xi, ...,Xm) to the set coordination code. 

Remark 1. Despite the apparent difference between set coordination and empirical coordination, it 
would be interesting to study whether set coordination can be expressed in terms of empirical coordi¬ 
nation (under either zero error, or asymptotically zero error criteria). As we will see in the proof of 
Theorem [II the known converse techniques for empirical coordination extend to set coordination. 

3.2 One-way coordination capacity 

A two nodes network is characterized by two alphabet sets Xi and X 2 for inputs and two action sets Ti 
and T 2 - For each pair of inputs (xi,X 2 ) £^ 1 X ^:2 we have a permissible action set Ax^^x 2 ^ Ti x T 2 - 
We are given some p(xi,X 2 ) on the inputs. Assume that there is a one-way communication link of 
limited rate R from node one to node two, as depicted in Fig. [TJ 

Definition 2. Coordination is achievable with one-way communication rate R with block length n if 
there are encoding and decoding functions 

V2:[2^^]x 

such that is coordinated with {Xf,X'!f) where the message M = 

£{Xf). 

A rate R is said to be one-shot achievable if it is achievable with a code of block length n = 1, and is 
said to be asymptotically achievable if it is achievable with a code with for some arbitrarily large block 
length n. We use C to denote the maximum one-shot achievable rate, and C to denote the supremum 
of the asymptotically achievable rates. 

Remark 2. Computing the one-way capacity C is in general a difficult problem. For instance, consider 
the special case ofYi being a constant random variable, i.e. |Ti| = 1, and T 2 = A’l. Further assume 
2/2 G Axj^^x2 if and only if y 2 = xi; in other words, the task of node one will be to communicate 
xi to node two. Given some p{xi,X 2 ), the problem reduces to a zero-error version of the Slepian- 
Wolf problem. The minimum rate for zero-error source coding with side information is known as 
Witsenhausen’s zero-error rate and is open in general m Practical code designs for this problem can 
be found in 
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4 A general lower bound 


One can use the fact that zero-error coordination is a more stringent condition than vanishing error 
coordination to derive the following lower bound on the asymptotic coordination capacity. Let V be 
the class of all p{yi,y2\xi^X2) such that p{yi,y2\xi^X2) > 0 only if (^1,^2) € Then we have: 

Theorem 1 . If node one observes Xi and node two observes X2 and they want to produce Yi and Y2 
respectively then minimum rate required from node one to node two is bounded from below by: 

max min min I(F]Xi\X2). ( 5 ) 

(j(xi,X 2 ): qiyi,y 2 \xi,X 2 )&'P q{f\xi,yi)-. 

q{xi,X2)=0 if p{xi,X2)=0 Iq{X\,Yr,Y2\F,X2)=0 

Proof of TheoremUl Coordination capacity depends only on the support of p{xi,X2), i.e. {{xi^X2) '■ 
p(xi, X2) > 0 }. Furthermore, if the support of a pmf q{xi,X2) is smaller than the support of p(xi, X2), 
the coordination capacity for q{xi,X2) will be less than or equal to that for p{xi,X2)- Therefore, it 
suffices to show that the coordination capacity for the input pmf p{xi,X2) is bounded from below by 

min min I{F]Xi\X2). ( 6 ) 

p{yi,y2\xi,x2)£V p{f\^i,yi)- 

Ip{Xi,Yv,Y2\F,X2)=0 

Take an arbitrary set coordination code of length n. Consider the converse given in [181 Appendix G] 
for interactive channel simulation under the empirical coordination constraint. Consider the special 
case of one round of communication r = 1 . This converse can be exactly mimicked for the set 
coordination code until equation ( 102 ). In equations ( 102 ) onwards in [181 Appendix G], it is argued 
that the joint pmf of (Liq, Y2Q, Xiq, X2q), where Q is the time sharing random variable, is close to the 
desired joint pmf p{yi,y2,xi,X2) that we wish to simulate. By contrast, in set coordination, we have 
that (Yiq,T2q) G ^Xiq,X2q with probability one. Therefore, p{yiQ,y2Q\xiQ, X2 q) G V. Therefore, the 
last step in the proof can be completed, and we get that the transmission rate is greater than or equal 
to I{F; Xiq\X2q). Since p{yiQ,y2Q\xiQ, X2 q) G F, we get that the coordination rate is greater than 
or equal to 


min min /(F;Ai A 2 ). 

p{yi,y 2 \xi,x 2 )£V p{fW,yi)- 

Ip{Xi,Yv,Y2\F,X2)=0 

(7) 


□ 


Corollary 1 . If X2 and Yi are constant random variables, random variable F will satisfy Xi —>■ Y —>■ 
12 CLnd we would like to minimize I{Xi;F). The choice of F = Y2 will he optimal and we get that the 
coordination capacity is greater than or equal to 

max min /(Ai;!^). 
q{xi) p{y 2 \xi)&'P 

We will see later that this lower bound is tight. 

5 No side information at node two 

In this section we restrict to the special case of a two nodes network, with node two having no 
observation, i.e., 1^21 = 1 . Without loss of generality we assume that p{xi) > 0 for all xi throughout 
this section. In this case we can make some simplifications as follows: we can drop X2 from Ax.,^x2 s-iid 
define 

Axi = {2/2: ^yi G Ti, (yi,y2) g A XiX 2 }- 

Notice that if node two chooses a proper action y2 from 4 ^;^, by definition there will exist a proper 
action yi for node one. Since node one knows xi and the message sent to node two as well as the 
decoding strategy of node two, action y2 and hence yi can be found by this node. 
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5.1 One-shot achievable rates 


We would like to compute the minimum size of the alphabet set of message M sent from node one to 
node two, such that node two can choose an action y2 G 3^2 where y2 € ■ In other words, assuming 

that M G Al, we would like to minimize I Ad I as much as possible. The one-shot capacity is minimum 
of log I Ad I. 

Characterization in terms of graph covers: The one-shot capacity can be also expressed in 
terms of the size of minimum graph cover [ 2 ]. Before proceeding, we need some definitions: 

Definition 3 (Coordination Graph). Consider a bipartite graph Q = (A’i, 3 /’ 2 )? where by this notation 
we mean that the vertices in one part being indexed by elements of Ati and the vertices of the other 
part indexed by elements of T2- An edge is drawn between xi G Xi and y 2 G T2 if and only if 
2/2 G Ax^. We call this a coordination graph. Both the one-shot and asymptotic coordination capacity 
are characterized by the coordination graph. Hence we sometime denote the one-shot and asymptotic 
coordination capacity by C{G) and C{Q) respectively. 

Definition 4 ([!]). Consider a bipartite graph Q = (A’i,T 2 )- A cover set for Xi is a subset <S C T 2 
such that for each xi G Ai, there exist some 2/2 G <S such that xi and 2/2 are connected. 

Observe that the minimum length of message from node one to node two, |Ad| is equal to the size 
of minimum cover for Xi. This is because given a cover set S, node one can simply send the index 
of 2/2 in the cover set <S that is connected to xi. Conversely, given any strategy by node one, one 
can produce a cover set by putting together the outputs 2/2 corresponding to different values of the 
message M. 

Finding minimum cover set for graph Q = (^1,3/2) is equivalent to solving following integer linear 
programming; 


IP(^) = min ^ 0(2/2) 
y2&y2 

s.t. Vxi G Xi «(2/2) > 1 

y2.y2'^^i 
a{y 2 ) G {0,1}. 

Here C{G) = log2 IP(f?) is the minimum message length in terms of bits. 

5.2 Asymptotic achievable rates 

In the asymptotic version of the problem, we fix some block length n. The first node observes some 
sequence xf and needs to convey it to node two in a way it can produce 2/2 such that 2/2i G A^^^ for 
i £ [n]. Then, one can see that the solution to this problem is size of minimum cover for a bipartite 
graph on Xf and with two sequences x^ and 2/2 connected to each other if and only if 2 / 2 * G 
for i £ [n]. This graph can be expressed as G^, i.e., the tensor product of the graph for the one-shot 
case, G, with itself by n times. Then the minimum required rate for block length n is log(IP(^®"'))/n, 
and the limit of this when n goes to infinity is the asymptotic coordination capacity is equal to C{G), 
i.e., 

C( 0 ) = log lim (IP(^®^) 1 /’^). ( 9 ) 

The above limit exist by the Fekete’s lemma because the sequence of log(IP( 0 ®”)) is superadditive 
[ 3 ]. In fact, for any two arbitrary graphs Gi and G2) we have 

\P{Gi®G2)<\P{Qi)\P{Q2), ( 10 ) 

because one possible way to find a cover set for Gi <S> G2 is to first find a cover set for Gi and for G2 , 
and then take the Cartesian product of these two cover sets as a cover set for Gi 0 G2- 
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Authors in [ 3 ] also consider the above asymptotic version of the problem and show that C{Q) = 
log2 LP(^), where LP(^) is the relaxation of linear programming given in equation Q as follows: 

LP(^) = min ^ 0(2/2) 
y2&y2 

s.t. Vxi € Ai «(y2) > 1 

y2-y2'^^i 

0(2/2) > 0 . 

Motivated by the Hide and Seek problem of [Ij, Lovasz provides a combinatorial proof for the above 
relation in [ 6 ]. In order to prove 

C{G) = log lim (lP(gl®-)V-) = log LP(g), ( 12 ) 

n^oo 

firstly, it is clear that the above is relaxation of the linear program given in equation ([ 8 ]). Therefore 
LP(^) < IP(^). By using the dual problem of linear programming, Lovasz shows that 

\-P{Gi(^G 2 ) = \-P{Gi)\-P{g 2 ). ( 13 ) 

Thus IP(^®'^) > LP(^®'^) = LP(^)” for all n. Hence C{G) > log(LP(^)). To show the other direction, 
log(LP(^)) > C{G), Lovasz provides an elegant combinatorial argument using a greedy algorithm and 
the dual of the linear program. We refer the readers to [B] for the details. 

Remark 3 . While computing the minimum cover C{G) is an NP-complete problem (it is the sixth 
problem of Karp’s 21 NP-complete problems C{G) can be calculated in polynomial time, as it is 

the solution of a real linear program. 

It is known that C{G) can be arbitrarily large while C{G) is arbitrarily small IWf . This happens 
for the mail-order problem of Slepian, Wolf, and Wyner m- On the other hand, C{G) = C{G) holds 
for two families of “interval graphs and forests” that are defined in m- 

We now make the observation that this special case of the coordination problem is nothing but the 
“Hide and Seek” problem of McEliece and Posner [ 3 ] . Consider the following source coding problem. 
Let Ai be a set of natural numbers. Alice observes some number xi € Ai. We have a certain list of 
properties such as a number being even, being divisible by five, being a prime number, etc. Given 
some xi E Ti, Alice can find a subset of properties that are satisfied by xi, e.g., if xi = 5 , it is both 
prime and divisible by 5 . The goal of Alice is to inform Bob of at least one of these valid properties; 
thus the goal is not to inform Bob of xi, but one of its valid properties. The question is the minimum 
amount of communication needed from Alice to Bob to accomplish this task. It is not difficult to see 
that this problem is identical to the two node coordination problem with no side information at node 
two: T2 can denote the set of properties and can contain the list of properties that xi has. 

Connection to game theory: The solution to the Hide and Seek problem can be expressed in 
terms of the Nash equilibrium of the following zero-sum game: let player one (hider) choose xi E 
and player two (seeker) choose some 2/2 € 3 ^ 2 - Player one has to give player two one dollar if 2/2 E 
otherwise the payoff is zero. This game is called Hide and Seek because player one hides and player 
two seeks player one. We refer the readers to [ 3 ] for details. 

The fact that the above source coding is related to this game may come as a surprise. This is due 
to the fact that the commonly used achievability proofs and converses in information theory are not 
based on the Nash value of games. However, we observe that 

• The key feature of capacity regions and upper bounds thereof are essentially additive regions, 
meaning that they expand by a factor of n, when evaluated on n independent repetitions of a 
problem (see [ 5 ] for a discussion). The Nash value of the above zero-sum repeated game also 
has the additivity property, as the expected value of the total payoff is equal to the sum of the 
expected value of the payoffs in the individual games. Therefore, it is quite possible that rates of 
a capacity region have characterizations in terms of Nash value of carefully constructed games. 
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• One can draw simple operational connections between the communication problem and the Hide 
and Seek game. Take an arbitrary code in the communication problem. In the communication 
problem, the seeker (player two) can ensure that y 2 G Axi if he receives R bits from the hider. 
However, in the game setup, there is no communication link. Nonetheless, the seeker can still 
guess the R message bits and win the game with probability at least 2“^, regardless of the value 
of xi- Therefore, the seeker has a strategy that gives him a payoff of 2~^ regardless of the action 
of the hider. This gives a lower bound on the Nash value of the game in terms of minimum value 
of R. This lower bound is tight by the result of [1]. 


5.2.1 Connection to the Rate Distortion theory 

While elegant, Lovasz’s proof is combinatorial. Fortunately, the asymptotic capacity C can be found 
using standard information theory arguments as in [3]. A formal way to do so is to express the the 
problem in terms of a zero-error rate distortion problem. Consider a distortion measure d : Ti x T 2 
{0,1} where d{xi,y 2 ) = 0 if and only if xi and y 2 be connected, i.e., y 2 G When xi and y 2 are 
not connected, we can set the distortion to an arbitrary positive value, say d{xi,y 2 ) = 1. 

Let R{p{xi),D) be the standard rate distortion function for distortion function d{-,-) when the 
source Xi has pmf p{xi). Furthermore, let Ro{p{xi), D) be the zero-error rate distortion function, 
which is the minimum (asymptotic) rate which can guaranty average distortion less than or equal to 
D with probability one, i.e., with probability of excess distortion being zero. Then, it is easy to see 
that Ro{p{xi), D) at Z) = 0 is equal to Cg because E[d{X^,Y.^)] < 0 implies that then d(x”,?/ 2 ) = 0 
for all pairs (x",?/^) where ^(x",?/^) > 0. 

Observe that Ro{p{xi), D) depends only on the support ofp(xi), i.e., the values of xi wherep(xi) > 
0, and not on the exact values of probabilities p{xi). Furthermore, it is clear that R{p{xi),D) < 
Ro{p{xi),D) since in Rq{p{xi), D) we ask for exactly zero probability of exceeding the distortion, 
whereas in R{p{xi), D) we ask for an asymptotically vanishing probability of excess distortion. There¬ 
fore, assuming that p{xi) > 0 for all xi, we have 

Ro{p{xi),D) > m8ixR{q{xi),D). 
gwi) 

Interestingly, the above inequality holds with equality [71 Thm 4.2]: 

Rnipixi), D) = maxR(q(xi), D) = max min I(Xi:Y 2 ). (14) 

q(xi) q{xi) p{y2\x\):¥.d{X\ ,Y2)<D 

Now, let us specialize this result to our coordination problem, when distortion D = 0. Let 

R = {p{y 2 \xi) : p{y 2 \xi) > 0 only if y 2 G 


Then, the asymptotic coordination capacity is equal to 


C(^) = max min /(Xi;l 2 )- 
q{xi) piy2\xi)&P 


(15) 


An alternative expression for C{Q) follows from [71 Cor. 3.7], where it is shown that 
rate-distortion function has the following characterization at zero distortion D = 0: 


the standard 


R{p{xi),0) 


— min 

q{xi) 



xi:d{xi,y2)=0 


(16) 


Then from equation (1141) . we have 


CiG) = Ro{p{xi),0) 


= — min max log 
q{xi) j/2ey’2 


( ^ 9 (^ 1 )) 

xi:d{xi,y2)=0 


= max — max log 
q{xi) y2&y2 


xi:d(xi,y2)=0 


= max min Iq{Xi]Y 2 ), 
q{xi) p(j/2|a:i)G'P 


(17) 


Where Iq{Xi] Y 2 ) is the R&yi mutual information of order zero, defined in Section [21 The last equality 
holds because for any q{xi), from the definition of Io{Xi] Y 2 ) we have that 


miri Io{Xi;Y 2 ) = - log ( max V q{xi)). 
p{y2\xi)eP y2&y2 

Xi:d(xi,y2)=0 

Remark 4. Curiously, equation (ITTl) is also equal to the zero-error feedback capacity of a point-to- 
point channel, when it is positive. This expression is also related to the sphere-packing bound for error 
exponents (see for instance fIM Eq. (8)]). It would be interesting to find operational interpretations 
for these facts. 


5.2.2 Characterization in terms of Renyi mutnal information 

So far, we have mentioned three characterizations for C{Q): one in terms of a relaxed linear program 
by Lovasz, and two as follows; 


C{g) = max min /o(-^i;^ 2 ), 
q{x\) p{y2\xi)&'P 

(18) 

= max min /(Xi;l 2 ). 

q(x\) p{y2\xi)&'P 

(19) 


It is desirable to provide connections between various characterizations. Our first result states that the 
expression of Lovasz’s linear program can be also understood in terms of Renyi mutual information, 
which to best of our knowledge is new. In particular, we use that Lovasz’s linear program to show the 
following theorem; 

Theorem 2. Assuming p{xi) > 0 for all xi, we have 


C{G)= max min /oo(-^i; 1 ^ 2 )- (20) 

q{xi) p{y2\xi)eV 

Remark 5. Since mutual information of order a is a non-decreasing function in a, equations dlHl)- 
(EoD imply that for any a G [0, 00 ] we have 


C{G) =max min IaiXi;Y 2 ). 

q{xi) p{y2\xi)eV 

Proof of Theoreml^ Using equation (|3D, we need to show that 


C{Q) = max min log > max p{y 2 \xi) 

qixi) p(y2\xi)&'P Xi:q{xi)>0 ^ 


= min log > maxp{y 2 \xi) 

p(y2\xiYV XI 


Let 0 ( 2 / 2 ) = max p( 2 / 2 |xi). Then, we would like to show that 


x\ 


C{g) =log mm 0 ( 2 / 2 ). 

p{y2\xiYV ^ 


( 21 ) 
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Thus, C(Q) = logLP 2 (^) where 


LP2(S) = min^a(y2) 

y2 

s.t. a{y2) >p{y2\xi), Vxi 

'^P{y2\xi) = 1 , p{y2\xi) > 0 

y2 

P{y2\xi) = 0 if 1/2 xi, 

where by y 2 xi we mean that y 2 ^ Ax^ ■ Now, recall Lovasz’s linear programming formulation: 
C{Q) = log LP(^) where 


LP(^) = min 0 (^ 2 ) 
y2 

s.t. Vxi G Ai 0 ( 2 / 2 ) > 1 

y2'’y2'^xi 

0(2/2) > 0. 

Thus, we only need to show that the two LPs are equivalent. The LP given in equation (1221) has 
more variables than the one given in equation (I23p . Using a standard Fourier-Motzkin elimination on 
variables p{y 2 \xi), one can see that the former LP reduces to the latter LP. Hence we are done. 

□ 

Our aim was to show the connection between Lovasz’s linear program and the Renyi mutual 
information. But it is possible to provide an algebraic proof for Theorem [2] (see [S] for another 
algebraic proof). 

Second proof of Theorem Let 


Fa{q{xi)) = min Ia{Xi-,Y 2 ). 

v{y2\xi)&V 

Since Renyi mutual information Ia{X;Y) is a non-decreasing function of a, we have that 

Fi{q{xi)) < Foo{q(xi)) Vg'(xi). 

Furthermore, using the fact that p{xi) > 0 for all xi we have that 

Fooiqixi)) = log I ^niaxp(?/ 2 |xi) | 

\ y2 ^ ) 

does not depend on q{xi). Therefore, it suffices to show that at the q{xi) that maximizes Fi{q{xi)), 
we have that Fi{q{xi)) > Foo{q{xi)). 

A theorem by Shannon |23l Thm. 2] studies the pmf (/(xi) that maximize F’i(g'(xi)). This theorem 
implies that the joint pmf that obtains maxq( 3 ,^) Fi{q{xi)) satisfies q{x, y) = c - q{x)q{y) for some fixed 
c when q{x,y) > 0. This implies that I{X-,Y) = Iao{X]Y) = log(c). Hence, 

Foo{q{xi)) = min Ioo{Xi-,Y 2 ) < log(c) = maxFi(g'(xi)). 

v{y2\^l)&'P <?(a:i) 


□ 
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5.3 Connections with strong coordination 

Take some arbitrary p{y 2 \x\). Then, consider the following channel simulation problem: Alice ob¬ 
serves a sequence x”, unknown to Bob. The goal of Bob is to sample a sequence from the pmf 
YYi=iP{y 2 i\xii)- To achieve this, Alice can send nR noiseless bits to Bob, i.e., the communication 
rate is R. This is the problem of simulating the memoryless channel p{y 2 \xi) via a noiseless link of 
limited rate using the strong coordination criterion. Alice and Bob may share common randomness, 
independent of x”, at some limited rate Rq. 

The channel simulation problem implies set coordination if p{y 2 \xi) is such that p{y 2 \xi) > 0 only 
if y 2 E As discussed in the introduction, strong coordination with infinite shared randomness 

Rq = oo \s related to the empirical coordination (at least in the vanishing error formulation). It turns 
out that the connection between strong and empirical coordiantion exists here in the zero error case 
as well. Observe that Empirical coordination, itself, is related to set coordination. 

The above zero-error strong coordination problem has been studied in [3], where it is shown that 
the minimum rate R, when Rq = oo, \s equal to O Theorem 24]: 

log(y^maxp(y2|a:i))- 

y2 

Even though not mentioned by the authors of [3], the above relation is nothing but Ioo{Xi;Y 2 ) for a 
p{xi) where p(xi) > 0,Vxi. Also in Section [3l III.G] (the section on Weak simulation and reversibil¬ 
ity), formulas similar to the ones given in the first proof of Theorem [2] (both of the linear programs) 
appear, even though Theorem [2] is considering a different problem. Although, the linear program of 
equation m appears but the works of McEliece and Posner, or Lovasz are not cited. 

6 Two nodes with side information 

Assume that node two has some inputs X 2 , but that node one is aware of that, i.e., H{X 2 \Xi) = 0. 
As in the previous section, we can ignore Yi in our analysis. Let us define 

{y2 ■ 3i/i G Ti,(yi,y2) g A,,^x2}- 

If node two chooses a proper action y 2 from by definition there will exist a proper action yi for 

node one. Since node one knows X 2 and the message sent to node two as well as the decoding strategy 
of node two, action y 2 and hence yi can be found by this node. 

Without loss of generality we assume that p{x 2 ) > 0 for all X 2 , throughout this section. 

Definition 5. Take some arbitrary X2 G X2 and eonsider the coordination problem where X2 = X2 
is fixed and known to everybody, i.e., instead of the joint pmf p{xi,X 2 ) we eonsider the joint pmf 
q{xi,X 2 ) = p{xi\x2)1[x2 = X 2 ]. Since X 2 is assumed to be fixed, this problem falls into the class of 
problems considered in Section\^ and using Definition\^ a bipartite graph can be associated to it. We 
denote this graph by Q^*, which defined on (A’i,T 2 ); where Xi = {xi : p(xi|x 2 ) > 0}. 

Theorem 3. The one-shot eoordination eapacity for joint pmf p{xi, X 2 ) where H{X 2 \Xi) = 0 is equal 
to 

max C (03,2). 

X2 

The asymptotic coordination capacity for joint pmf p{xi,X 2 ) where H{X 2 \Xi) = 0 is equal to 

max C (03,2). 

X2 

Remark 6. This result implies a cutset bound for general coordination networks. If we divide the 
set of nodes into two groups, say, nodes {1, 2,..., k} in one group and nodes {k + l,k + 2, ..., m} in the 
second group, we can write a cut-set bound as follows: we assume two super-nodes that have access 
to X[fc] and and need to make actions and y[k+i:m] respectively. We also assume that a 
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genie provides the inputs of the second group to the first super-node. Then, the minimum total 

communication rate from super-node one to super-node two is bounded from below by the corresponding 
bound given in the above theorem for such a two node scenario. 


Proof of Theorem 0 The equation for one-shot case follows directly from the definition of Qx 2 , the 
fact that coordination needs to hold for all values of X 2 , and X 2 is known by both the nodes. 

It remains to show the result for the asymptotic case. Using the one-shot result and applying it 
to the n-letter version of the problem, the minimum coordination rate for codebooks of length n, will 
be equal to 

- maxCiGx^), 

n ^ 

where Qx^ = ^ 0:21 ® ^ 2:22 ® ••• ® ^x^n i® tensor product of the graphs for indices 1 to n. Therefore, 
the one-way communication capacity is equal to 


Thus, we need to show that 


Using the fact that logC'(^) 


lim — maxC(0xr}). 
n^oo n x^ ^ 


lim — m.axC{Qx'ii) = maxC'(^a; 2 )- 
n—>-00 n X 2 ^ X 2 


= IP(a) > V.P{Q)=\ogC{g) and LP(ai ^>^ 2 ) 

-maxC(t/a;^) > logmaxLP(t/2;n)V”' 

^ rr>Tl ^ 

^2 2 

n 

= logrnaxJjLP(a2:2j^^" 

i=i 

= logmaxLP(t/a;2) 

= max log lPiGx 2 ) 

X2 

= maxC{Gx2)- 

X2 


(24) 


LP(^i)LP(^ 2 ), we have 


(25) 


For the reverse direction we prove that for a given e > 0, there exist a natural number N such 
that for all n > iV, we have 

\P{Gx^,)<mQx-){l + eT, Vx^. (26) 

This equation would then imply that 


-max(7(^3,") = - maxlog IP(^2,5) 
n x^ ^ n X 2 ^ 

< - maxlog LP(^j;n)-p log(l-K e) (27) 

n X 2 ^ 

= maxC{Gx 2 ) + log(l -h e). 


It remains to show equation (|26p . Without loss of generality assume that X 2 = {l,2,...,r} for some 
natural number r. Take an arbitrary sequence xlf. Let Uj be the number of indices i such that X 2 i = i 
for i £ [r]. Then Ui = n and the tuple {nijn, n 2 /n, ...^nrjn) indicates the type of the sequence X 2 . 
Because for any two graphs G\ and G 2 we have 


IP(I?i ® Gfi) = IP(^2 ® ^ 1 ), 

we get that \P(Gx^) = IP(<^i=i From equations (flOl) and (fT^ , we have that 

r 

ip(^x5) <n'p(^ro, (28) 

i=l 
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(29) 


r 

= X\i.P{QT^). 

i=\ 

Taking logarithms from both sides and dividing by n, we get 

-iogip(e.j) < ^^iog(ip(ar')'/"o, 

n ^ n 

2=1 

i log LP(^,.) = V - log(LP(eif"OVn.) 
n ^ n 

2=1 

= ^^log(LP(a*)), 

2=1 


Where in equation ([3T]l we used equation (fT3]l . 

From Lovasz’s result in equation (I12|) . we know that 

lim |P(C^)iM = LP(eii). 

ni—>oo 


(30) 


(31) 


Thus, given any <5 > 0, for each i € [r], if n* > W for some sufficiently large threshold Ni, we have that 
log(IP(0f’”*)^/”'“) is within 6 interval of log(LP(^j)). Compare equations ([3(1 and (l3T|) . If n* > Ni, 
we have that the corresponding terms in the sum are within S interval of each other. If n* < Ni, by 
letting n go to infinity, we can make rii/n < Ni/n as small as we want. This would complete the 
proof. □ 


Remark 7. We show that the general lower bound given in Section is tight in this case. Consider 
that X 2 is a function of Xi. Then, similar to Corollary [Jl when Xi —)■ F —)■ >2 and X 2 is a function 
of Xi, we have 

I{F-,X,\X2) = I{FY2-,X,\X2) > I{Y2-,Xi\X2). (32) 


Therefore, we know that 


max min 7(12; ^i|-^ 2 ) 
q{xi,X2) p{y2\xi,X2)e'P 


(33) 


is a lower bound for the coordination capacity, where the maximum is over all q{xi,X 2 ) such that 
q{xi,X 2 ) = 0 if p{xi,X 2 ) = 0. Observe that 


max min I{Y 2 ] Xi\X 2 ) > mccx. max min 7(12; 1^2 = 2 : 2 ) 

q{xl,X2)p{y2\x^,X2)&V X2 q{x\_\x2) p{y2\xi,X2)e.'P 

= maxC'(t?a;2), 


(34) 

(35) 


where the maximum in the second equation is over q{xi\x 2 ) where g'(xi|x 2 ) > 0 only if p{xi\x 2 ) > 0. 
Thus, max 3;2 C{Qx 2 ) is a tight lower bound to the eoordination capacity. 


7 Linear Coordination 

In this section we will study a special case of set coordination problems, where actions and inputs are 
constrained by linear equations. More specifically in a network with m nodes, we assume that the 
inputs Xi and output Yi are all vectors (of possibly different lengths) in a given field F. We say that 
the nodes are coordinated if 


NlXi + K 2 X 2 + • • • + KmXm + Km+lYi + • • • + K2mXm — 0, (36) 

for some matrices Ki, i ^ [2m]. 

One motivation for this model comes from linear control systems. Suppose that nodes are con¬ 
trollers and inputs are disturbances to the system. Controllers should undo the disturbance by pro¬ 
ducing proper actions. 
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Example 2 . Linear coordination is a generalization of linear network coding. To see this, for instance 
consider a network of m nodes, where node one has message of k hits for node m — 1, and node two 
has message of length I bits for node m. Then, we can write this as a linear coordination problem 
consider that 


Kl = —K2m-1 = 


h-kxk 

0 


K 2 = —K2m = 


(e+k)x{e+k) 


0 0 

0 lexe 


{i+k)x{£+k) 


( 37 ) 


While other Ki’s are zero. First k bits of Xi are the message for node m — 1 , and other i bits are 
zero. In first X2 first k bits are zero and rest of them are the message for node m. 


7.1 Linear coordination capacity 

For a linearly constrained coordination problem, we can define linear or non-linear codes. In a linear 
code, all encoding and decoding operations are linear, i.e., the transmitted messages are constructed 
linearly from the inputs, and the output actions are reconstructed linearly from the messages and the 
inputs. On the other hand, a non-linear code allows for non-linear encoder and decoders. 

For simplicity, we only consider the two nodes network with one-way communication of Section 
[321 We assume that the inputs of nodes are column vectors Xi G F''^, X2 € ^ distributed according 

to some joint distribution. The outputs of the two nodes are also assumed to be column vectors 
Yi € F*bl2 G The nodes are coordinated if 

KiXi + K2X2 + K3Y1 + KiY2 = 0 , ( 38 ) 

for some fixed matrices Ki, K2, K3 and K4. These four matrices are assumed to have the same number 
of c rows. The number of columns of Ki, K2, K3 and K4 are ri,r2,si and S2 respectively. Similar 
to Section O we further make the simplifying assumption that node two has no input (r2 = 0 ), or 
equivalently K2 = 0. 

The one-shot (non-linear) coordination capacity of the communication link from node one to node 
two, C{Ki, K3, K4), is defined as before for the permissible action sets 

= {(2/1,^2) : Kixi + K^yi + Kiy2 = 0 }, xi € Ti. 

On the other hand, the one-shot linear coordination capacity Cl{Ki, K3, K4^) is defined as follows: 

Definition 6 (One-shot linear coordination capacity). The message M generated from node one is 
assumed to be a linear combination of coordinates of Xi, i.e., M = S'Xi for some matrix S in 
where t is the number of symbols that are transmitted. Actions Yi and Y2 are constructed linearly 
according to Yf = AXi and Y2 = BM. The goal is to find matrices S, A and B such that equation 
(I 38 p holds, while t (the number of rows of matrix S) is minimized. The minimum value oft is called 
the one-shot linear coordination capacity and denoted by Cl{Ki, K3, K4). 

Because linear codes are a special case of non-linear codes, we have that C < Cl log |F|. 

In an asymptotic coordination problem, we have multiple inputs and outputs Xii,Yii,Y2i for i G [n]. 
We say that the nodes are coordinated if 

KiXu + K^Yu + K^Y2^ = 0 , Vz G [n]. ( 39 ) 

Alternatively, if we make an overall vector of inputs Xi by concatenating column vectors Xu, X12,..., Xi„, 
and similarly for Yi and Y2, we can express equation (| 39 p as follows: 

{Inxn<^Ki)Xi + {Inxn<^K3)Yi + {Inxn<^K4)Y2 = 0 , ViG[n]. ( 40 ) 

Therefore, we can make the following definitions: 

Definition 7 (Asymptotic linear coordination capacity). The asymptotic linear coordination capacity 
is defined as 

Cl{^K\, K3, Kfi) — lim CLi^Inxn ® I^ixu ® X3, I^ixn ® Xfi). 

n—>-oo n 

Similarly, the asymptotic non-linear coordination capacity can be expressed as 

C(^Ki, K3, — lim C i^Inxn ® Irfixn ® X3, Irfixn ® Xfi). 

n—>-00 n 
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7.2 Linear one-way coordination capacity 

In this section we study one-shot and asymptotic linear capacity for the one-way communication setup. 

Theorem 4 . Let subspace 'V be the linear span of all vectors xi € such that p{xi) > 0 . Then we 
have CL{Ki,K3,Ki) = Cl{Ki,Ks,K^) = f{Ki,K3,Ki) where 

f{Ki,K^,K4) = min dim 

s.t. KirTJmiKs)®^ (41) 

^ C Jm{K4). 

Here the minimum is over linear subspaces ^ that satisfy the given constraints, © is the Minkowski 
sum and llm(-) is the image operator. 

Proof of Theorem\^ We first prove that Cl{Ki, K^, K4) = f{Ki,Ky,, K4) and then prove that Cl{Ki, K^, K4) 
Cl{KuKs,K4). 

Proof of Cl{Ki, K^, K4) = f{Ki,K3,K4): The coordination constraint can be written as 


KiXi + KsYi + K4Y2 = KiXi + K3AX1 + K4BSX1 
= {Ki + K 3 A + K 4 BS)Xi = 0 

Thus, {Ki + K3A + K4BS)xi = 0 for all xi where p{xi) > 0 . Therefore, the necessary and sufficient 
condition for this equation to hold is that 

(Ki + K3A + K4BS)-r = 0 . ( 42 ) 

We have that K3A'Y C ^111(1^3) and K4BS'Y C 3xn{K4B) = ^. Then since B has t columns we have 
dim' 2 ^ < t. Therefore, equation (H 2 ]) implies that equation (|ini holds for some appropriate ^ and 
Cl{K 4 ,K 2 ,K 3 )> f{K 4 ,K 2 ,K 3 ). 

To show the other direction, assume that there exist a vector space ^ C 3m.{K4) such that 
KiY C Dim(it'3) © and dim"^ = t. We will find appropriate matrices A,B and S. Let {uj} be a 
basis for Y. Equation (H 2 ]) is true if and only if {Ki + K3A + K4BS)vi = 0 . Now let Qi = KiVi. Since 
KiY C Jm(iL3) © '^ we can find vectors Wi G Jm(iL3) and ri^^ such that qi = WiP ri. 

Now we want to find matrix A such that Wi = K3Avi for all i. If we show [ui| • • • by V and 
[rcil • • • \ wi] by W. Then we should find A such that W = K3AV. Since Wi € Jm{K3) we can find 
matrix L such that W = K3L. Thus we need L = AV. Notice that since {vi} are a basis for Y, 
matrix V is full column rank. Therefore its rows span the full space. Let us denote the i-th row of 
L hy li, and i-th row of Ahy ai. We have to solve linear equation li = aiV where Oj is a vector that 
we need to find. Now since span of rows of V is the full space we can always find proper vectors a,. 
Hence we can find A such that W = K3AV. 

Now for finding matrices B and S (with t rows); we should have = K4BSvi. Take R= [ri| ■ ■ ■ \ri]. 
We need R = K4BSV. 

Since ^, we have that rank{R) < t. Since G Jm(iL4), we can find L' such that R = K4L'. 
Now, we want to prove that we can choose L' such that rank{L') < t. Notice that each columns of L' 
can be written as the sum of two vectors such that one of them lies in the kernel space of K4 and the 
other one is perpendicular to this space. Therefore L' = + L^. We have that K4L' = K4L'^ + K4L'2. 

The first part K4L'^ will vanish and K4L' = K4L2. Therefore, without loss of generality we can assume 
that 3 m{L') C Ker{K4)^ or Jm(L') n Ker{K4) = 0 . Next, it is known for arbitrary matrices A and 
B that rank{AB) = rank{B) — dim(Jm(H) n Ker{A)). Thus, 

rank{R) = rank{L') — dim(Dim(L') n Ker{K4)) = rank{L'). ( 43 ) 

Thus rank{L') = rank{R) < t. 

Now we should have L' = BSV. Again notice that V is full column rank and rank of L' is equal 
to t. Therefore there exist t rows that can produce all rows of L'. Choose S such that t rows of SV 
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can produce L' . Now if we denote rows of SV by si, • • • , st and rows of L' by • • • , since is in 
the span of si, • • • , st, we can find real numbers hij such that = Yhj bijSj if we take B = [hj], then 
we have L' = B{SV). This completes the proof. 

Proof of Cl{Ki, K^, K 4 ) = Cl{Ki, K 3 , K 4 ): It suffices to show that 


-Cl{I2x2 ® Ki,l2x2 < 2 > Ks, I2x2 ® K4) = Cl{Ki, K3, K4). 


This is because the above equality can be used inductively to show that for any n, which is a power 
of two, we have 

Cl {In X n ® Bi , Inxn ® , In xn ® B4 ) — Cl {B\ , , K 4 .). 

n 

Therefore Cl{Bi, K3, B4) = Cl{B i,B3,B4). 

Consider that node one observe two vectors Xu and X12 and then sends M = S 
second node where 

5 = 


Xn' 

X12 


to the 


^2 

^3 ^4 


Node one produces 


and node two produces 


Xn' 



A2 

'Xn 

.^ 12 _ 


A3 

A4 

Xi 2 _ 


V2i' 

= BM = 

Bi B2 

'Si S2 

Xii' 

X22_ 


B3 B4 

S3 S'4 

X12 


Coordination constraint for first letter gives us 

(Xi + K2A4 + BsiBiSi + B2S3))Xn + {K2A2 + X 3 (Bi 52 + B 2 S 4 ))Xi 2 = 0 . ( 44 ) 


A similar condition holds for the second letter. Since Xu and X12 are independent, we get 


(Xi + K2A1 + + B 2 S 3 ))Xu = {B2A2 + Bs{BiS2 + B254))Xi2 = 0 . 


Equivalently, 


(Xi + K 2 A 1 + B^iBiSi + B2S3))y = {B 2 A 2 + B3{BiS2 + B2S4))y = 0. 

We claim that without loss of generality, we can make the following two assumptions: 

• We can assume that columns of Bi are perpendicular to the kernel space of B3. To see this, 
observe that each columns of Bi can be written as the sum of two vectors such that one of them 
lies in the kernel space of B3 and the other one is perpendicular to this space. This gives us a 
decomposition of matrix Bi as Bi = Bn + Bi2 where B^Bn = 0 . Since K^Bn vanishes, only 
B3Bi2 remains and we may assume that columns of Bi are perpendicular to the kernel space of 

Bs- 

• we can assume that SiZ = 0 for i = 1 , 2 , 3,4 and for all vectors z € where X"*" is the linear 
subspace perpendicular to X. 

Notice that for every matrix S we can find matrix S' with same dimension such that Sv = S'v 
for all u G X; and S'z = 0 for all z G This is because we can define a linear function like 
S' : F’’! —7> F* by determining effect of this function on a basis. If {ui, • • • , be an orthonormal 
basis for X and we expand this to an orthonormal basis for F^ like {ui, • • • ,vi,zi, - • • Cri-i}, 
then we can define S'{vi) = Svi and S'{zi) = 0 . It is only effect of matrices on X in important 
for us, and we can consider that S'z = 0 if z G 
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Each column of can be written as the sum of two vectors such that one of them lies in the 
image of K2 and the other one perpendicular to image of K2. This gives us a decomposition of matrix 
as it's = K^i + -ff32, where all columns of are in image of K2 and all columns of K^2 are 
perpendicular to this space. Now since node one knows B and S it can choose A[ and A2 such that 

K2A[ = K2A1 + + B2S3), 

K2A'2 = K 2 A 2 + K^i{BiS2 + B2Sa). 

Using the above equations, we get that 

{Ki + K2A[ + K^iiBiSi + B2S^))y = 0 
{K2A'2 + K:,2{BiS 2 + B2Si))y = 0 

The second equation implies that {B1S2 + B2Si)'f' = 0 since there is no common vector except zero 
in images of K2 and K^2 (all columns of K^2 were perpendicular to the image space of K2). A similar 
argument for the second letter shows that {B^^Si + BiS^)'f = 0. 

On the other hand, from the assumption that = 82^^ = 8 ^ 1 ^^ = 84,'^^ = 0 , we have 

that {B182 + B28i)'y-^ = {B^8\ + B^8z)'y-^ = 0 . This fact, in conjunction with {B182 + B28i)'f' = 
{B^8i + Bi8'i)y = 0 , implies that B182 + B284 = B381 + B483 = 0 . 

Now 


'B481 + B283 

B182 + B284 


'B481 + B283 

0 

B:i8i + B4S3 

B382 + B484 


0 

Bs82 + B484 


So rank{BM) = rank{Bi8i + B283) + rank{B^82 + 5454). Besides, rank{BM) < 2 t. Therefore 
either rank{Bi8i + B283) < t 01 rank{B^82 + Bi8i) < t. Thus, one can do one letter coordination 
with rate less than or equal to t. Therefore 

-jCL{l 2 x 2 ® Ki,l 2 x 2 ® ATs, /2x2 ® K 4 ) > Cl{Ki,K^, K 4 ). 

and block coding cannot help. 

□ 


7.3 Linear and non-linear coordination capacities 

Theorem 5 . Assuming Jm{K4) C iJm(iCi), we have that C = C = C^loglFl = C'/.loglFj. In other 
words, linear strategies are optimal and block coding does not help decrease the non-linear coordination 
rate. 

Proof. From Theorem 01 we know that Cl = Cl- We know that Cl log |F| > C > C. The asymptotic 
coordination capacity C is equal to logLP for the linear program of equation m, on a graph Q 
constructed as follows: ^ is a bipartite graph with nodes indexed by Ti on one part, and by T2 on the 
other part. Vertex xi is connected to y2 if and only if there is some yi such that Kixi+K'4yi+K4y2 = 0 . 

Let LP^ be the dual of the linear program given in equation (llip . This linear program is given in [ 6 l 
page 2 ] and consists of variables in the interval [ 0 , 1 ]. While the original LP involved a minimization and 
was asking for a fractional covering, the dual linear program involves maximizing a linear expression 
and can be understood as a fractional packing linear program. If we restrict the variables of the dual 
program to integers in { 0 , 1 }, we get a lower bound on LP^. We denote the answer to this integer 
program by IP^. It is shown in 0 page 2 ] that IP^ is the maximum number of vertices in Xi whose 
neighbor sets in the bipartite graph are disjoint. 

To sum this up, we always have the following chain of inequalities: 

Cl log |F| > C > C = log LP = log LPt > log IPl. 


We show that 


log I pi > CiloglFl, 


( 45 ) 
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which implies that all of the above inequalities are equality. Observe that both Cl log |F| and log IP^ 
are one-shot expressions and can be computed from the graph Q (rather than its tensor products). 

Assume that two vertices xi and x'l have a common neighbor like ?/2- Then there exist yi,y'i such 
that 

Kixi + K-^yi + K4y2 = Kix\ + K3y[ + ^' 4^2 = 0 . ( 46 ) 

Hence A'i(xi — x[) G Dim(A'3). Furthermore, Kixi, Kix'i G Dim(iFi). Therefore, to show that IP^ > N, 
it suffices to find vectors vi, such that 


Vi G 3 m(A:i), 

Vi-Vk ^ JmiKs), Vi ^ k, 


( 47 ) 


Let ^ be the vector space with minimum dimension in Theorem Q. The dimension of ^ is equal 
to Hence there are distinct vectors in ^. We claim that the set of vectors in satisfy 

both conditions of equation ( 1471 ) . This would imply that IP^ > and gives us equation (| 45 l) . 

First, observe that C 3 m.{K4) C Jm(Ari). Hence, the first condition of (| 171 ) is clearly satisfied. 
To show the second condition, observe that is a vector space with minimum dimension such that 
Jm(iFi) C 3m{K3)(B^. We claim that this implies ^ nDim(Ar3) = { 0 }. Otherwise if « G ^ fl Jm(Ar3), 
we can expand n to a basis for ^ like {u, ui, ■ ■ ■ Let be the linear span of the vectors 

(ui,--- ,uc^_i). Then is a subspace of '2xn{K4) that satisfies 31:1(773) © = 3 m( 773 ) © 

Therefore, we can decrease dimension of ^ which is a contradiction. This completes the proof. □ 


7.4 Extensions to mnltiple-terminal 

It is possible to extend the result in linear coordination to certain multi-terminal scenarios. For 
instance, consider a network with a broadcast channel topology depicted in Fig. [21 where node one 
observes Ai G F^^ and sends SAi to node two and TXi to node three. All three nodes produce 
outputs, but nodes two and three have no inputs. Coordination constraint requires that KiXi + 
77411 +775I2 + 77613 = 0 . In a linear code, we have that li = 73 Xi, and the outputs of nodes two and 
three are constructed linearly from their received messages, i.e., Y2 = C{SXi) and I3 = D{TXi) for 
some matrices C and D. This gives us the equation 

( 77 i + 774H + K^CS + KeDT)Xi = 0 . ( 48 ) 



Fj 


Figure 2: Linear coordination with broadcast topology. 

Thus, ( 77 i + K4B + K^CS + KqDT)xi = 0 for all xi where p{xi) > 0 . Therefore, similar to 
previous parts we define Y to be the linear span of all vectors xi G F’’^ such that p{xi) > 0 . Hence 

{Ki + K^B + K^CS + KeDT)^ = 0. ( 49 ) 

Now with an argument similar to the one given in the proof of Theorem (H we have following region 
for sizes of 5 't^xri and T^^xn- The pair of (fi,t2) is valid in one-shot case if and only if 

ti > dim'll t2 > dim'll 
KiY C 3m774 © © "^2 

C 3m(775), ^2 + 3m(776). 
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Next, consider a MAC channel where we assume that node one and two observe two independent 
vectors Xi and X 2 respectively (see Fig. [3]). These two nodes send messages SXi and TX 2 to node 
three. All three nodes produce outputs linearly from their observations and their received vectors. 
We want to find minimum number of rows of S and T. Coordination constraint is KiXi + K 2 X 2 + 
KiYi + K 5 Y 2 + AeTs = 0. Assuming that W = AAi, Y 2 = BX 2 and Ts = CSXi + DTX 2 , we get 
that 

Ai Ai + K 2 X 2 + KiYi + K 5 Y 2 + KeYs = (Ai + K^A + KeCS)Xi + (As + A 5 A + AgAT) As = 0. 


Xi 



^2 


Figure 3: Linear coordination in MAC. 

Now since Xi and As are independent both parenthesis should be zero. And these terms are 
exactly same as the ones for the two nodes case. Therefore, if we denote the number of rows of S and 
T by(ti,ts)) then such a pair is valid in one-shot case if and only if 

ti > dim'll ts > dim '^1 

AiA C am(A4) ©^ 1 , 

A2Acam(A5)©'^s, 

^1,^2 Y 3 mAg. 
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